Robot, control device, and information processing device

ABSTRACT

A robot includes an input detection portion, a motion detection portion, and a control portion. The input detection portion is configured to detect an input given from an operator to a robot body. The motion detection portion is configured to detect a motion by using the input detection portion, the motion being given by the operator. The control portion is configured to execute a motion instruction associated with the motion detected by the motion detection portion.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to a robot and the like.

Description of the Related Art

In recent years, robots that perform collaborative work with humans havebeen developed.

For increasing the efficiency of collaborative work, it is important fora human to give a motion instruction appropriately and easily to a robotwhen they are performing the collaborative work.

For example, Japanese Patent Application Publication No. 2019-93522proposes a robot system in which a motion instruction is given to arobot. In the robot system, a voice-based motion instruction is given tothe robot for causing the robot to work together with a peripheraldevice for performing work on a workpiece.

When a human and a robot perform collaborative work in a position wherethey are close to each other, it is convenient if a voice-based motioninstruction is given to the robot, as disclosed in Japanese PatentApplication Publication No. 2019-93522.

However, in a case where the robot is an industrial robot in particular,there are other industrial machines installed around the robot andproducing noise. The noise from the other machines, the operating soundproduced by the robot itself, and the work sound produced in the work(e.g., machining) performed by the robot may prevent the voice-basedmotion instruction from being accurately recognized. If the human had togive an extremely loud voice to the robot for reliably giving thevoice-based motion instruction to the robot, or move from the vicinityof the robot to a silent position for giving a voice to the robot, theefficiency of the collaborative work would be lowered.

Thus, in the field of working robots, a method of accurately giving amotion instruction to a robot has been desired.

SUMMARY OF THE INVENTION

According to a first aspect of the present invention, a robot includesan input detection portion, a motion detection portion, and a controlportion. The input detection portion is configured to detect an inputgiven from an operator to a robot body. The motion detection portion isconfigured to detect a motion by using the input detection portion, themotion being given by the operator. The control portion is configured toexecute a motion instruction associated with the motion detected by themotion detection portion.

According to a second aspect of the present invention, a control deviceincludes a motion detection portion and a control portion. The motiondetection portion is configured to detect a motion by using an inputdetection portion, the input detection portion being configured todetect an input given from an operator to a robot body, the motion beinggiven by the operator. The control portion is configured to cause therobot body to execute a motion instruction associated with the motiondetected by the motion detection portion.

According to a third aspect of the present invention, an informationprocessing device includes a motion detection portion and a controlportion. The motion detection portion is configured to detect a motionby using an input detection portion, the input detection portion beingconfigured to detect an input given from an operator to a robot body,the motion being given by the operator. The control portion isconfigured to output a motion instruction associated with the motiondetected by the motion detection portion.

According to a fourth aspect of the present invention, a control methodincludes detecting an input given from an operator to a robot body,detecting a motion given by the operator, and causing the robot body toexecute a motion instruction associated with the detected motion.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view schematically illustrating an externalappearance of a robot of a first embodiment.

FIG. 2 is a diagram illustrating electrical connection of the robot ofan embodiment.

FIG. 3A is a block diagram illustrating an internal configuration of acontrol device of an embodiment.

FIG. 3B is a block diagram illustrating an internal configuration of ajoint of the robot of an embodiment.

FIG. 4 is a diagram illustrating a system configuration of the robot ofthe first embodiment.

FIG. 5 is a flowchart of the whole of procedures 1 and 2 of anembodiment.

FIG. 6 is a flowchart of a user-motion analysis process of anembodiment.

FIG. 7A is one example of a displayed image of an embodiment.

FIG. 7B is another example of a displayed image of an embodiment.

FIG. 8A is an example of a time-series torque detection signal caused bya user motion and detected in a first joint.

FIG. 8B is an example of a time-series torque detection signal caused bythe user motion and detected in a second joint.

FIG. 8C is an example of a time-series torque detection signal caused bythe user motion and detected in a third joint.

FIG. 9A is an example of a time-series torque detection signal caused bythe user motion and detected in a fourth joint.

FIG. 9B is an example of a time-series torque detection signal caused bythe user motion and detected in a fifth joint.

FIG. 9C is an example of a time-series torque detection signal caused bythe user motion and detected in a sixth joint.

FIG. 10 is a graph of a motion analysis waveform in an embodiment.

FIG. 11A is a diagram illustrating one example of user motion.

FIG. 11B is a diagram illustrating another example of user motion.

FIG. 11C is a diagram illustrating still another example of user motion.

FIG. 12A is a diagram illustrating one example of user motion.

FIG. 12B is a diagram illustrating another example of user motion.

FIG. 12C is a diagram illustrating still another example of user motion.

FIG. 13 is a schematic diagram for illustrating auser-motion-and-motion-instruction associating process.

FIG. 14 is a flowchart of a procedure 3 of an embodiment.

FIG. 15 is a diagram illustrating an external appearance of a robot armof a robot of a second embodiment.

FIG. 16A is a diagram illustrating one example of user motion of thesecond embodiment.

FIG. 16B is a diagram illustrating another example of user motion of thesecond embodiment.

FIG. 17 is a diagram illustrating inputting a user motion in the secondembodiment.

FIG. 18 is a diagram illustrating a pop-up of the second embodiment,displayed when a user motion is deleted.

FIG. 19 is a diagram illustrating a list of user motions and motioninstructions, of the second embodiment.

DESCRIPTION OF THE EMBODIMENTS

Hereinafter, some embodiments of the present invention will be describedwith reference to the accompanying drawings. Specifically, thedescription will be made for a robot that performs collaborative worktogether with a human, a control device that controls the robot thatperforms collaborative work together with a human, a method ofcontrolling the robot that performs collaborative work together with ahuman, and the like.

Note that in the drawings that will be referred to in the description ofthe following embodiments, a component given an identical referencesymbol has an identical function, unless specified otherwise.

First Embodiment

Robot

In a first embodiment, an industrial robot having a six-axis-controlrobot arm will be described. However, the present embodiment of thepresent invention is not limited to this type of robot, and the presentinvention can be applied for various types of robot as long as each ofthe robots can perform collaborative work together with a human in aposition where they are close to each other. In addition, in the presentembodiment of the present invention, the number of joints of a robot andthe number of axes of the robot are not limited to specific values. Forexample, the present embodiment can be applied for a verticallyarticulated robot, a SCARA robot, a parallel link robot, a Cartesiancoordinate robot, a dual-arm robot, or the like. In addition, the robotmay have various actuators and sensors in accordance with thecollaborative work with a human.

FIG. 1 is a perspective view schematically illustrating an externalappearance of a robot 500 of the first embodiment. As illustrated inFIG. 1 , the robot 500 includes an articulated robot arm 100 that holdsand assembles a workpiece W into a product, a control device 200 thatcontrols the robot arm 100, and a system control panel 300 that isconnected to the control device 200. The articulated robot arm 100 is asix-axis-control robot arm, and has a hand (end effector) 102 connectedto a leading end of the robot arm 100.

The robot arm 100 includes a base portion 103 fixed to a workbench, aplurality of links 121 to 126 that transmit displacement and force, anda plurality of joints 111 to 116 that pivotably or rotatably link theplurality of links 121 to 126 to each other. For causing the robot 500to perform work, the control device 200 controls the position andposture of the robot arm 100 by driving the joints 111 to 116.

When the robot 500 performs collaborative work together with a human,the control device 200 detects that the human touches the robot arm 100(or inputs a user motion, by using an output signal from a torquedetection device of each of the joints 111 to 116; and analyzes(detects) the way of touching (i.e., touch pattern). That is, in thepresent embodiment, the torque detection device functions as a touchdetection portion that detects a touch of an operator to the robot body,and a user-motion analysis portion that serves as a motion detectionportion detects a motion of the operator by using the output signal fromthe torque detection device. In other words, the control device 200analyzes (detects) the motion of the human when the human touches therobot arm 100. After detecting the touch pattern (motion) of the humanobtained when the human touches the robot arm 100, the control device200 controls the motion of the robot in accordance with a motion pattern(instruction) that has beau stored in advance and associated with thedetected motion.

The system control panel 300, which is connected to the control device200, is used as an input/output device when an operator teaches a motionto the robot 500 or sends an instruction to the robot 500. The systemcontrol panel 300 includes an input portion, through which an operationinstruction is inputted; and a display portion which displays theinformation on the state of the robot arm and the like. In addition, thesystem control panel 300 may include an emergency stop switch. In thepresent embodiment, the system control panel 300 can also be used forcausing the control device 200 to learn a touch pattern (motion) of ahuman, and for causing the control device 200 to associate a motionpattern with the motion of the human and store the motion pattern.

FIG. 2 is a diagram illustrating electrical connection of the robot 500.FIG. 3A is a block diagram for illustrating an internal configuration ofthe control device 200. FIG. 3B is a block diagram for illustrating aninternal configuration of joint of the robot arm.

In FIG. 2 , a power-supply device 101, the control device 200, thesystem control panel 300, and the joints 111 to 116 are illustrated.

The power-supply device 101 supplies electric power to the whole system.For example, the power-supply device 101 supplies alternating-currentpower to a power-supply circuit (see FIG. 3A) of the control device 200.The power-supply circuit of the control device 200 converts thealternating-current power to direct-current power, and supplies thedirect-current power to a power-supply circuit (see FIG. 3B) of each ofthe joints 111 to 116, via a power supplying line 145. Note that anothermethod other than the above-described power supplying method may be usedfor supplying power to each component.

The control device 200 is connected to the joints 111 to 116 via acommunication line 146, through which the control device 200 cantransmit a control signal. The control device 200 controls the positionand posture of the robot arm 100 by controlling the motion of the joints111 to 116.

As illustrated in FIG. 3A, the control device 200 includes thepower-supply circuit, a communication control device, a trajectorycreation device, a peripheral I/O control portion, a memory, and a CPU.In addition, the control device 200 includes a user-motion analysisportion 501, a user-motion storage portion 502, auser-motion-and-motion-instruction associating portion 503, and amotion-instruction storage portion 504. The operation of these functionblocks will be described later, associated with the robot-motion controlthat is performed in accordance with a user motion.

The joints 111 to 116 illustrated in FIG. 2 have the same or a similarinternal configuration. Thus, in the present embodiment, the internalconfiguration of the joint 116 will be described, as an example, withreference to FIG. 3B.

The joint 116 includes a joint control portion 109, a position detectiondevice 30 a, a motor 30, a reduction gear 31 that reduces rotationtransmitted from the motor 30, and a torque detection device 20 thatdetects the torque applied to the joint 116.

The joint control portion 109 is a control circuit that includes thepower-supply circuit, a communication control device, a computingdevice, a serial communication device, an AD converter, a currentdetection device, and a motor driver.

The communication control device is connected with the control device200 via the communication line 146, and sends/receives a signal to/fromthe control device 200, for example, on a cycle of 10 milliseconds. Thecommunication control device transmits a signal to the control device200. For example, the signal is an output signal from the positiondetection device 30 a, an output signal from the torque detection device20, a signal into which the output signal from the position detectiondevice 30 a or the torque detection device 20 has been processed, or asignal that indices information on the state of the joint and containserror or alarm information. When the communication control devicetransmits a signal into which an output signal from the positiondetection device 30 a or the torque detection device 20 has beenprocessed, the output signal from the position detection device 30 a orthe torque detection device 20 may be processed appropriately by thecomputing device or the AD converter. The communication control devicereceives an instruction for controlling the operation of the motor 30,from the control device 200. When the communication control devicereceives an instruction for controlling the operation of the motor 30,from the control device 200, the joint control portion 109 drives themotor 30 by using a motor driver.

The motor 30 is a servo motor, and may be a brushless DC motor or an ACservo motor. The motor 30 is fastened to the link 125 via bolts or thelike, and the power from the motor 30 is transmitted to a reduction-gearinput shaft of the reduction gear 31. The position detection device 30 ais directly mounted on a rotary shaft of the motor 30, and generates apulse signal in accordance with the rotation of the motor 30 and outputsthe pulse signal. Note that a brake unit may be disposed between themotor 30 and the position detection device 30 a, as necessary, forkeeping a posture of the robot arm 100 while the power is off. Theposition detection device 30 a may be an optical device or a magneticdevice, like a general-purpose rotary encoder.

The reduction gear 31 is suitably a strain-wave-gearing reduction gear,but may be another reduction gear. The reduction gear 31 includes thereduction-gear input shaft that receives the power from the motor 30, areduction-gear fixing portion that holds the reduction gear 31 itself,and a reduction-gear output shaft that outputs torque whose speed hasbeen reduced. If the reduction gear 31 is a strain-wave-gearingreduction gear, the reduction-gear input shaft includes an ellipticalcam and an elastic bearing. In addition, an inner circumferentialportion of the ring-shaped reduction-gear fixing portion and an outercircumferential portion of the cup-shaped reduction-gear output shaftthat is an elastic member have teeth. The number of teeth of the innercircumferential portion is different from the number of teeth of theouter circumferential portion, and the teeth of the innercircumferential portion and the teeth of the outer circumferentialportion mesh with each other. When the elliptical reduction-gear inputshaft is rotated by the motor 30, the reduction-gear output shaft thatis an elastic member elliptically deforms, and the reduction-gear outputshaft and the reduction-gear fixing portion mesh with each other at bothends of the major axis of the ellipse. Thus, when the reduction-gearinput shaft is rotated by the power from the motor 30, thereduction-gear output shaft meshes with the reduction-gear fixingportion while deforming in elliptical shape, and the link 126 (notillustrated in FIG. 3B) is rotated relative to the link 125 and thetongue detection device 20, which are on the reduction-gear fixingportion side.

The torque detection device 20 is a torque sensor that uses an opticalencoder or the like to detect the amount of deformation of a structure,and converts the amount of deformation to a torque value. The torquedetection device 20 is disposed between the link 125 and thereduction-gear fixing portion, and detects the torque applied to thejoint. Note that a magnetic extender or an electrostrictive force sensormay be used as the torque detection device 20.

Next, with reference to FIG. 4 , a system configuration of the robot 500will be described.

The robot 500 includes a CPU 1201 that serves as a computing unit, a ROM1202, a RAM 1203, a storage portion (HDD) 1204, a recording-disk drive1205, and various interfaces 211 to 216.

The CPU 1201 is connected with the ROM 1202, the RAM 1203, the HDD 1204,the recording-disk drive 1205, and the various interfaces 211 to 216 viaa bus 217. The ROM 1202 is a non-transitory recording medium, and storesa base program such as a BIOS. The RAM 1203 is a storage device thattemporarily stores results of a computing process performed by the CPU1201.

The HDD 1204 is a storage unit that stores various data, which isresults of a computing process performed by the CPU 1201; and stores aprogram 330 that causes the CPU 1201 to execute various types ofprocess. The CPU 1201 executes the various types of computing process,depending on the program 330 recorded (stored) in the HDD 1204, which isa recording medium. The recording-disk drive 1205 can read various typesof data and a program stored in a recording disk 331.

In the present embodiment, the user-motion storage portion 502 and themotion-instruction storage portion 504, which are illustrated in FIG.3A, are achieved in the HDD 1204. In addition, the program for achievingthe function blocks, such as the user-motion analysis portion 501 andthe user-motion-anti-motion-instruction associating portion 503illustrated in FIG. 3A, are included in the program 330 stored in theHDD 1204.

The interface 211 is connected with the system control panel 300 that auser can operate.

The interface 212 is connected with the torque detection device 20,which outputs a torque detection value to the CPU 1201 via the interface212 and the bus 217. The interface 213 is connected with the positiondetection device 30 a, which outputs a position-detection output signalto the CPU 1201 via the interface 213 and the bus 217.

The interface 214 is connected with a display portion 311 that displaysvarious images; the interface 215 is connected with an external storagedevice 312, such as a rewritable nonvolatile memory or an external HDD.The interface 216 is connected with a servo control device 313.

Depending on a driving instruction sent than the CPU 1201, the servocontrol device 313 calculates the amount of current to be supplied tothe motor 30, supplies the current to the motor 30, and performs jointangle control on the joints 111 to 116 of the robot arm 100. Forexample, when performing force control, the CPU 1201 controls the motor30 via the servo control device 313, and causes the motor 30 to drivethe joints 111 to 116 so that a torque detection value (i.e., outputsignal from the torque sensor unit) of each of the joints 111 to 116becomes equal to a target torque value. In addition, when performingposition control, the CPU 1201 outputs instruction data, used fordriving the motor 30 and indicating the amount of control on therotation angle of the motor 30, to the servo control device 313,depending on an output signal from the position detention device 30 a,at predetermined intervals via the bus 217 and the interface 216.

Robot Motion Control by Using User Motion

For causing the robot 500 to perform a motion such as a predeterminedmotion that has been taught, the trajectory creation device of thecontrol device 200 creates control information, and the joints 111 to116 are driven, depending on the control information, so that theposition and posture of the robot arm 100 is controlled.

However, in a case where the robot 500 performs collaborative worktogether with a human, the robot 500 does not necessarily repeat apredetermined motion (for example, a robot motion that has been taught).Instead, while performing collaborative work together with a human, therobot 500 can analyze a user motion when the human touches the body ofthe robot 500, interpret the user motion as a motion instruction, andchange the motion of the robot 500.

Note that the user motion represents regularity, style, similar type,and the like of the touch obtained when the human touches the body(typically, the robot arm 100) of the robot 500. For example, theregularity, the style, the similar type, and the like are determined,depending on a length of touch time in which the human touches a robothousing with a hand (that is, the human inputs a user motion), astrength of force, a direction of force, a cycle, the number of touchesin a predetermined time, a touch position, change in touch positions,and the like. Then, the determined regularity, style, and similar typeare identified as a user motion. That is, at least one of the lengths oftouch time, the strength of force applied in the touch, the direction offorce applied in the touch, the cycle of the touch, the number oftouches in a predetermined time, the touch position, and the trace oftouch position is analyzed, so that the user motion is detected.

FIGS. 11A to 11C are schematic diagrams for illustrating a plurality ofpatterns of user motion. FIG. 11A illustrates a user motion “long touch”in which a human keeps touching a link of the robot arm for a long time.FIG. 11B illustrates a user motion “double touch” in which a humantouches a link of the robot arm, successively two times, each performedin a short time. FIG. 11C illustrates a user motion “triple touch” inwhich a human touches another link of the robot arm, successively threetimes, each performed in a short time. Thus, the control device 200 canidentify these three types of user motion, for example, by analyzing thenumber M of times a human has touched the robot.

In addition, as illustrated in FIGS. 12A to 12C as examples, the abilityfor identifying user motion can be increased by analyzing not only thenumber M of times a human has touched a robot, but also a touch-timelength Tn, a touch-time interval In, a force strength (contactpressure), and the like.

Note that the subscript n of Tn and In indicates an index of each touchevent.

When a user motion is identified, a motion instruction that has beenassociated in advance with the user motion and stored is read, and thecontrol device 200 executes or change a motion of the robot inaccordance with the motion instruction. For example, the motioninstruction instructs one of deceleration, acceleration, stop, restart,position control start, force control start, and error reset start, or acombination thereof. However, the motion instruction may instructanother motion.

For allowing the robot to change the motion in accordance with a usermotion, the control device 200 of the robot 500 of the presentembodiment performs processes in accordance with the followingprocedures.

Procedure 1: User-Motion Storage Process

The control device 200 analyzes and learns the way of touching (usermotion) in which a human gives a motion instruction by touching therobot 500, and the user-motion storage portion 502 stores a similar typeof the analyzed user motion. Note that when the control device 200analyzes and learns a user motion, the control device 200 may use aso-called machine learning method that uses the neural net or the like.

Procedure 2: Motion-Instruction Storage Process

The motion-instruction storage portion 504 stores each of motioninstructions given by a human touching the robot 500. As previouslydescribed, for example, the motion instruction instructs one ofdeceleration, acceleration, stop, restart, position control start, forcecontrol start, and error reset start, or a combination thereof.

The user-motion-and-motion-instruction associating portion 503 storesthe relationship between a similar type of a user motion and acorresponding motion instruction.

Note that the restart may be a resume motion. In the resume motion,after the control device 200 determined that the robot 500 could notmove for some reason, and stopped the robot 500, the control device 200resumes the robot 500 that has been in the abnormal state. In the resumemotion, the robot 500 is controlled so that a predetermined portion (inthe present embodiment, the link 126 that is a leading and of the robotarm 100) is positional at a predetermined position. The predeterminedposition is set such that the leading end of the robot arm 100 of therobot 500 does not interfere with its surroundings, and is called aresume position or an origin position. If the restart instruction isgiven, the robot 500 moves the link 126 to the resume position or theorigin position, to return from the abnormal state. In addition, anoperator may associate an emergency-stop instruction, by which theoperator immediately stops the robot 500 for some reason, with a usermotion; and may cause the control device 200 to execute theemergency-stop instruction.

The force control start may be a start of a motion that brings aworkpiece into contact with another workpiece. In this case, apredetermined hand is attached to the robot 500, and the workpiece isbrought into contact with the other workpiece in accordance with theforce detected by the torque detection device. The position controlstart may be a start of a motion that conveys a workpiece. In this case,a predetermined hand is attached to the robot 500, and the workpiece isconveyed by the hand. Such instructions for the force control start andthe position control start may be used when the robot 500 is restarted.

Procedure 3: Motion Change Process in Accordance with User Motion

When a human touches the robot 500 in collaborative work, theuser-motion analysis portion 501 detects a user motion by using anoutput signal from the torque detection device 20 of each joint andinformation on the user motion stored in the user-motion storage portion502. Note that the user-motion analysis portion 501 can use a so-calledlearned model produced by using the neural net or the like.

If a user motion is detected, the user-motion-and-motion-instructionassociating portion 503 identifies information on the motion instructioncorresponding to the detected user motion, from among the information onthe motion instruction stored in the motion-instruction storage portion504.

The control device 200 accepts the identified information as a motioninstruction, and then controls the motion of the robot 500 in accordancewith the motion instruction.

Description of Procedures

First, the procedures 1 and 2 will be described with reference toflowcharts illustrated in FIGS. 5 and 6 . FIG. 5 is a flowchart of thewhole of the procedures 1 and 2. FIG. 6 is a flowchart of a user-motionanalysis process included in the procedure 1.

As illustrated in FIG. 5 , in the procedure 1, a user-motion recognitionstart process is performed in Step S301. For example, Step S301 isstarted when an operator presses a start button of the system controlpanel 300, and a screen illustrated in FIG. 7A is displayed on thedisplay portion 311 of the system control panel 300. Note that thedisplay portion 311 may be a display device that only displays images,or may be a touch panel device that not only displays images but alsoreceives input data.

As illustrated in FIG. 7A, the screw contains a mode display portion201, a motion-registration-number display portion 202, amotion-registration cancel button 203, a motion storage button 204, anumber forward/backward button 205, an operation-state display portion206, and number input keys 207. In this step, the mode display portion201 displays a message “INPUT MOTION” for prompting an operator toperform an input operation of a user motion.

When the operator presses the motion storage button 204, Step S302 isstarted for storing a user motion, and a screen illustrated in FIG. 7Bis displayed on the display portion 311. Specifically, the mode displayportion 201 displays a message “ENTRY MOTION”, and themotion-registration-number display portion 202 displays anidentification number for distinguishing a user motion to be stored,from other user motions.

The operator inputs a user motion to be stored, by touching the robot500. For example, if the operator desires to store one user motion thatis to pat a side surface of the housing of the robot arm, twice, theoperator performs the one user motion. While the operator is performingthe user motion, a time-series output signal is transmitted from thetorque detection device 20 of each of the joints 111 to 116, to thecontrol device 200; and is stored in a memory of the control device 200.When the operator presses the motion storage button 204 again, thestorage of the time-series torque detection signal corresponding to theuser motion is completed.

In Step S303, it is determined whether the torque detection signalscorresponding to a predetermined number of user motions (desired typesof user motion) have been stored. For example, in a case where fivetypes of user motion are desired to be stored, if the torque detectionsignals corresponding to the five types of user motion have been stored(Step S303: YES), then the pros, proceeds to Step S304. However, if thetorque detection signals corresponding to the five types of user motionhave not been stored (Step S303: NO), then the process returns to StepS302 and stores another user motion. That is, Step S302 is repeateduntil the predetermined number of user motions are stored.

If the time-series torque detection signals corresponding to thepredetermined number of user motions are stored in the memory (StepS303: YES), then the process proceeds to Step S304 and the user-motionanalysis portion 501 analyzes each use motion. That is, the user-motionanalysis portion 501 analyzes the characteristics of the way of touchingof each user motion, depending on the torque detection signal; and theanalyzed characteristics are stored in the user-motion storage portion502.

FIG. 6 is a flowchart of the user-motion analysis process of Step S304.FIGS. 8A to 8C and 9A to 9C are graphs of the time-series torquedetection signals produced from one user motion and measured in therespective joints. The time-series torque detection signals are storedin the memory of the control device 200.

In Step S401, each of the time-series torque detection signals detectedby a corresponding torque detection device is subjected to a low-passfilter process. In the graphs of FIGS. 8A to 8C and 9A to 9C, the torquedetection signals detected by the torque detection devices areillustrated by dotted lines, and low-pass-filtered torque detectionsignals are illustrated by solid lines.

In Step S402, an offset removal process is performed. Specifically, theload applied to each joint of the robot anti is removed from thecorresponding low-pass-filtered torque detection signal of the joint.

In Step S403, a square operation process is performed on theoffset-removed torque detection signal of each joint.

In Step S404, a totalizing process is performed on the torque detectionsignals of the joints, on which the square operation process has beenperformed in Step S403. In the present embodiment, the torque detectionsignals of the six joints on which the square operation process has beenperformed are totalized.

The totalized result is referred to as a motion analysis waveform. FIG.10 illustrates an example of a graph of the motion analysis waveform.

In Step S405, a peak search process is performed for performing featureextraction, for identifying a user motion by using the motion analysiswaveform. First, a wave whose peak-to-peak value is larger than a presetthreshold is extracted from the motion analysis waveform. In the exampleof FIG. 10 , since the threshold is −3, two waves M1 and M2 areextracted.

Then, the area of each of the extracted waveforms is evaluated. Forsimplicity, a product of a half value of a peak-to-peak value and ahalf-value width in the time axis may be calculated. In the example ofFIG. 10 , a product S1 of a half value P1 of a peak-to-peak value and ahalf-value width T1 in the time axis is calculated for evaluating M1,and a product S2 of a half value P2 of a peak-to-peak value and ahalf-value width T2 in the time axis is calculated for evaluating M2.

Then, the product S1 calculated for M1 and the product S2 calculated forM2 are stored in the user-motion storage portion 502, as feature valuesof the user motion. That is, the products S1 and S2 are associated withan identification number displayed in the motion-registration-numberdisplay portion 202 of the display portion 311 of the system controlpanel 300, and stored in the user-motion storage portion 502.

Note that the feature values of the user motion are not limited to theabove-described products S1 and S2. For example, the feature values maybe a time interval between M1 and M2, peak values of M1 and M2, orinformation that identifies a joint in which the maximum value of thetorque detection signals has been measured. In addition, for obtainingthe feature values, fitting may be performed on a model waveform byusing the least squares method, and the residuals may be determined; orotherwise, waveform pattern recognition that uses the machine learningor the like may be used.

Referring back to FIG. 5 , after the characteristics of the user motionsare analyzed and the storage of the characteristics of each user motionis completed in Step S304, the process proceeds to Step S305 thatperforms a motion-instruction storage process.

In Step S305, a motion instruction, which is to be given to the robot bya human performing a user motion (i.e., a predetermined motion in whichthe human touches the robot), is stored in the motion-instructionstorage portion 504 of the control device 200. For example, the motioninstruction instructs one of deceleration, acceleration, stop, restart,position control start, force control start, and error reset start, or acombination thereof.

In Step S306, a user-motion-and-motion-instruction associating processis performed. FIG. 13 is a schematic diagram for illustrating theuser-motion-and-motion-instruction associating process. In theuser-motion storage portion 502, the feature values of the user motionsare associated with respective identifiers UM1, UM2, UM3, . . . , andstored. In the motion-instruction storage portion 504, the motioninstructions are associated with respective identifiers F1, F2, F3, . .. , and stored.

In Step S306, each user motion is associated with a corresponding motioninstruction, and is stored. Specifically, a table is created in theuser-motion-and-motion-instruction associating portion 503, and therelationship between a user motion and a corresponding motioninstruction is stored in the table.

Thus, an operator can resister the relationship between a user motionand a corresponding motion by using the system control panel 300.

When Step S306 is completed, the above-described procedures 1 and 2 arecompleted. Note that when Step S306 is completed, the display portion311 of the system control panel 300 may display a list on which eachuser motion is associated with a corresponding motion instruction, forallowing an operator to easily understand the relationship between theuser motion and the motion instruction.

Next, the above-described procedure 3, that is, a motion change processperformed in accordance with a user motion will be described.

When the robot 500 is performing a predetermined motion, for example, inaccordance with an operation program that has been taught, controlinformation is created by the trajectory creation device of the controldevice 200, and the joints 111 to 116 are driven, depending on thecontrol information, so that the position and posture of the robot arm100 is controlled. In the present embodiment, while the robot 500 isoperating, the user-motion analysis portion 501 monitors the torquedetection signal sent from the torque detection device 20 of each of thejoints. The user-motion analysis portion 501 detects a user motion byusing the information on user motion, stored in the user-motion storageportion 502.

When a user motion is detected, the user-motion-and-motion-instructionassociating portion 503 identifies a motion instruction corresponding tothe detected user motion, from among the motion instructions stored inthe motion-instruction storage portion 504. The control device 200accepts the identified motion instruction, and changes the motion of therobot 500 in accordance with the motion instruction.

With reference to a flowchart of FIG. 14 , processes performed in theprocedure 3 will be described.

After started up, the robot 500 executes Step S501 that is a systeminitialization process. The system initialization process includessequence control; and in the sequence control, internal variables areinitialized in an internal process, communication between internalcomponents of the robot 500 and between the robot 500 and an externaldevice is automatically established, and a power supply of eachcomponent is started up.

After the completion of Step S501, Step S502 is executed for setting acommand for causing the robot 500 to perform a predetermined motion inaccordance with an operation program. The control device 200 performsinteractive communication, periodically with the joints 111 to 116. Inaddition, the control device 200 sets a control command created by thetrajectory creation device, to an internal variable ExecCmd; andacquires state information from each joint.

After the execution of Step S502, Step S503 is executed for monitoring atorque detection signal measured by the torque detection device 20 ofeach joint. In Step S503, the user-motion analysis portion 501 performsthe user-motion analysis process (see FIG. 6 ) by using the torquedetection signal sent from each joint.

In Step S504, the user-motion analysis portion 501 determines whether ahuman has performed a user motion, by referring to the featureinformation stored in the user-motion storage portion 502.

If no user motion is detected (Step S504: NO), then the process proceedsto Step S506 while the control command that was set to the internalvariable ExecCmd in Step S502 is kept, and executes the control command.That is, since there is no user motion, the process does not change themotion and executes the current command.

On the other hand, if a user motion is detected (Step S504: YES), thenthe process proceeds to Step S505, and theuser-motion-and-motion-instruction associating portion 503 reads amotion instruction corresponding to the detected user motion, from themotion-instruction storage portion 504.

Then the process rewrites a control command corresponding to the motioninstruction that has been read, into the internal variable ExecCmd. Thenthe process proceeds to Step S506, and executes the rewritten controlcommand. In this case, since the user motion was detected, the motionhas been changed in accordance with the motion instruction correspondingto the detected user motion.

After the execution of Step S506, the process proceeds to Step S507, andthe information on the state of each joint of the robot arm is sent tothe control device 200 via the communication line 146.

Then, the process returns to Step S502, and the steps S502 to S507 arerepeated, for example, on a cycle of 10 milliseconds for thecollaborative work between the human and the robot 500.

As described above, in the present embodiment when a human and a robotthat performs collaborative work together with the human are performingcollaborative work in a position where they are close to each other: thehuman can accurately and easily give a motion instruction to the robot.In particular, when a human and a robot are manufacturing productsthrough collaborative work (for example, assembling components into aproduct, or machining such as grinding, cutting, or painting) in aposition where they are close to each other, the human can accuratelyand easily give a motion instruction to the industrial robot.

Second Embodiment

In the first embodiment, the user motion obtained when a human touches arobot in collaborative work is analyzed by using the torque detectiondevice disposed in each joint of the robot arm. However, the embodimentsof the present invention are not limited to this.

A second embodiment described below differs from the first embodiment inthat the user motion obtained when a human touches the main body of arobot in collaborative work is analyzed by using a touch panel attachedto the robot arm. That is, in the present embodiment, the touch panelfunctions as a touch detection portion that detects a touch of anoperator to the robot body, and the user-motion analysis portion thatserves as a motion detection portion detects a motion of the operator byusing an output signal from the touch panel. Hereinafter, the samefeatures as those of the first embodiment will be omitted in thedescription, and different features from the first embodiment will bemainly described.

The touch panel used in the second embodiment may be any touch panel aslong as the touch panel can be attached to the housing of the robot at aposition where a human can easily touch the touch panel. In addition,the touch panel may be fixed to the robot, or may be detachably attachedto the robot.

The touch panel may have any system as long as the touch panel candetect a touch of a human in the system. The touch panel may be attachedonto a display screen.

FIG. 15 is a diagram illustrating an external appearance of a robot armof a robot of the second embodiment. A panel holder 902 is disposed on aside surface of a housing of a robot arm 901, and a touch panel 904 canbe attached to the panel holder 902. Preferably, the touch panel 904 isdesigned so that an operator can perform user motions, such as longtouch, short touch, sequential touch, repeated touch, flick, swipe,pinch, and line drawing, by touching the touch panel 904 with a finger.Suitably, at least one of the lengths of touch time, the strength offorce applied in the touch, the direction of force applied in the touch,the cycle in the touch, the number of touches in a predetermined time,the touch position, and the trace of touch position is analyzed, so thatthe user motion is detected. With this analysis, at least one usermotion of long touch, short touch, sequential touch, repeated touch,flick, swipe, pinch, and line drawing can be identified. In the presentembodiment, the description will be made, as an example, for a casewhere the touch panel 904 includes a display screen 905 and can be usedalso as an operation panel, which can display the state of the robot armand through which informations can be inputted into the robot.

The touch panel 904 can communicate with the control device 200,wirelessly or via wire. If the touch panel 904 communicates with thecontrol device 200 wirelessly, the touch panel 904 may have a batteryand communicate with the control device 200 by using infraredcommunications, wireless LAN, Bluetooth (registered trademark), or thelike. If the touch panel 904 communicates with the control device 200via wire, the touch panel 904 and the robot arm may be connected witheach other via a connector 903, and a signal line and a power supplyline may be connected to the touch panel 904 and the robot arm.

Since the touch panel 904 can be used also as an operation panel, thetouch panel 904 may be used as a teaching pendant when teaching isperformed, or may be used like the system control panel 300 of the firstembodiment. However, when a human performs collaborative work in thevicinity of the robot, the touch panel 904 is used as a device forinputting a motion instruction by performing user motion. Thus, sincethe information processing device, such as a teaching pendant, throughwhich a motion instruction can be inputted from the outside isdetachably attached to a corresponding robot, an operator caninstantaneously find which teaching pendant corresponds to which robot.Consequently, the operator can accurately give a motion instruction tothe robot by using a corresponding teaching pendant.

In the present embodiment the way of touching (touch position, length oftouch time, cycle, and the number of touches in a predetermined time) inwhich a human touches the touch panel 904 and the trace of touchposition (change in touch position in time series) are analyzed as thecharacteristics of user motion.

FIG. 16A illustrates an example of a user motion that a human performsby touching the touch panel 904 for inputting a motion instruction. FIG.16B illustrates an example of another user motion that a human performsby touching the touch panel 904 for inputting a motion instruction.

Also in the present embodiment, as in the first embodiment, a usermotion is analyzed in advance before the robot performs collaborativework together with a human, and the characteristics of the user motionare stored in the user-motion storage portion. In addition, a motioninstruction is stored in the motion-instruction storage portion, inadvance. Furthermore, the relationship between a user motion and acorresponding motion instruction is stored in theuser-motion-and-motion-instruction associating portion, in advance. Forexample, the user motion illustrated in FIG. 16A is associated with amotion instruction that stops the robot arm, and the user motionillustrated in FIG. 16B is associated with a motion instruction thatrestarts the robot arm.

As an example of the characteristics of user motions, the trace of touchposition detected by the touch panel 904 is stored. Note that when thetrace (pattern) of touch position is inputted as a user motion, thedisplay of the mode display portion 201 of a display screen 905 ischanged to “PATTERN MOTION”, and a pattern input portion 906 isdisplayed, as illustrated in FIG. 17 . In addition, a pattern deletebutton 907 and a pattern storage button 908 are displayed in thevicinity of the pattern input portion 906. When an operator desires toinput a predetermined pattern in the pattern input portion 906 and storethe pattern, the operator touches the pattern storage button 908. If theoperator touches the pattern storage button 908, a pattern that theoperator inputs will be stored. First, the pattern input portion 906 iscleared, and then the pattern registration number 909 is changed forprompting the operator to input a new pattern. If the pattern deletebutton 907 is touched, the pattern having been inputted in the patterninput portion 906 is deleted for allowing the operator to input anotherpattern. If the pattern delete button 907 is touched, a pop-up 912 isdisplayed, as illustrated in FIG. 18 , for allowing the operator toconfirm the delete of the pattern.

If the operator desires to edit a pattern that has already beenregistered, the operator touches a motion edit button 910. Then thepattern registration number 909 struts to blink. If the operator touchesa number forward/backward button 911, the pattern registration number909 chance while blinking, and patterns having been registered in thepattern input portion 906 are displayed in a sequential manner. If theoperator touches the pattern delete button 907 in a state where apattern that the operator desires to edit is displayed in the patterninput portion 906, the pop-up 912 illustrated in FIG. 18 is displayed.

After that, if the operator touches a “YES” button, the registeredpattern is deleted. Then the operator can input a new pattern in thepattern input portion 906 for editing the pattern. The patternregistration number 909 blinks for allowing an operator to easilydistinguish a mode to check registered patterns by using the numberforward/backward button 911, from a mode to input a pattern. If anoperator has mistakenly touched the motion edit button 910, or if theoperator desires to merely check registered patterns, the operatortouches the motion edit button 910 again. Then the pattern registrationnumber 909 stops blinking.

As described in the first embodiment, the display screen 905 may displaya list on which each pattern is associated with a corresponding motioninstruction, for allowing an operator to easily understand therelationship between the pattern and the motion instruction. FIG. 19illustrates an example of the list on which each pattern is associatedwith a corresponding motion instruction. The pattern and thecorresponding motion instruction can be displayed sequentially bypressing the number forward/backward button 911. In the presentembodiment, each displayed pattern is provided with an arrow thatindicates how the pattern has been inputted. Note that if an operatortouches a portion in which a pattern is displayed, and then touches themotion edit button 910, the screen may be changed to the display screenillustrated in FIG. 17 and corresponding to the pattern, for allowingthe operator to edit the pattern. Thus, since the user motion and thecorresponding motion instruction are displayed in a list, an operatorcan easily understand the relationship between a pattern and acorresponding motion instruction. In addition, the display screen asillustrated in FIG. 19 may be used as part of an instruction manual ofthe robot. Note that the above-described display modes may be changed bypressing a button (not illustrated), or a physical button disposed onthe touch panel 904.

When a user motion of a human is actually detected, the trace of touchposition is detected by the touch panel 904, than the starting point,the slope, and the scale of the trace is corrected, and then thecorrected trace is compared to a trace that has been stored in a memory.Specifically, fitting process is performed on the corrected trace andthe trace stored in the memory, by using the least squares method; and adistance between both of the traces is calculated. If a variance isequal to or smaller than a preset threshold, it is determined that auser motion has been detected.

Note that when the characteristics of user motion are analyzed andlearned, a so-called machine learning method that uses the neural net orthe like can be used. In this case, for detecting a user motion while arobot is performing collaborative work, a so-called learned modelproduced by using the neural net or the like can be used.

In the second embodiment described above, when a human and a robotperform collaborative work, the human can accurately and easily give amotion instruction to the robot while performing the collaborative workin a position where they are close to each other. In particular, when ahuman and a robot are manufacturing products through collaborative work(for example, assembling components into a product, or machining such asgrinding, cutting, or painting) in a position where they are close toeach other, the human can accurately and easily give a motioninstruction to the industrial robot.

Modifications

The present invention is not limited to the above-described embodiments,and may be variously modified or combined within the technical conceptof the present invention.

For example, in the first embodiment, it is determined, in Step S303 ofthe flowchart of FIG. 5 , whether the torque detection signalscorresponding to desired types of user motion have been stored. However,the determination may be made in a different manner. For example, thereis a case in which a predetermined number of measurement samples(learning data) is collected by causing an operator to perform one typeof user motion, several times, for increasing the learning accuracy ofthe user motion. In this case, the process may determine in Step S303whether the same type of user motion has been performed thepredetermined number of times. In this case, in Step S303 that is a usermotion analysis-and-storage process, a learned model on user motion maybe produced by using the neural net and stored.

In addition, both of the torque detection device of the first embodimentand the touch panel of the second embodiment may be attached to anidentical robot and used as a mechanism for detecting a user motion. Inthis case, the torque detection device and the touch panel may be usedin parallel or alternatively, depending on situation.

The present invention can also be achieved by providing a program, whichperforms one or more functions of the above-described embodiments, to asystem or a device via a network or a storage medium, and by aprocessor, which is included in the system or the device, reading andexecuting the program. In addition, the present invention can also beachieved by using a circuit, such as an ASIC, which performs one or morefunctions.

In addition, the above-described various embodiments can be applied forany machine that can automatically perform expansion and contractionmotion, bending and stretching motion, up-and-down motion,right-and-left motion, pivot motion, or combination motion thereof,depending on information data stored in the storage device of thecontrol device.

Other Embodiments

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2020413019, filed Mar. 12, 2020, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. A robot system comprising: an input detectionportion configured to detect an input given from an operator to a robotbody; a motion detection portion configured to detect a motion by usingthe input detection portion, the motion being given by the operator; anda control portion configured to execute an operation instructionassociated with the motion detected by the motion detection portion,wherein, by an operation of the operator, the control portion isconfigured to cause a display portion to display an information on themotion of the operator and an information on the operation instructionassociated with the motion of the operator.
 2. The robot systemaccording to claim 1, wherein the input detection portion is configuredto detect the input by using a torque detection device provided in therobot body.
 3. The robot system according to claim 1, wherein the inputdetection portion is configured to detect the input by using a touchpanel attached to a robot arm of the robot body.
 4. The robot systemaccording to claim 3, wherein the touch panel is detachably attached tothe robot arm.
 5. The robot system according to claim 3, wherein thetouch panel comprises the display portion.
 6. The robot system accordingto claim 3, wherein the touch panel is configured to communicate withthe control portion by using at least one of infrared communications,wireless LAN, Bluetooth, and wire via a connector.
 7. The robot systemaccording to claim 1, wherein the motion detection portion is configuredto detect the motion given by the operator, by analyzing at least one ofa length of input time of the input, a strength of force of the input, adirection of force in which the input is applied, a cycle of the input,a number of times of the input in a predetermined time, a position atwhich the input is applied, and a trace of the position of the input. 8.The robot system according to claim 7, wherein the input is performed bythe operator touching the robot body.
 9. The robot system according toclaim 1, further comprising a user-motion storage portion configured tostore information on a characteristic of the motion, wherein the motiondetection portion is configured to detect the motion of the operator byusing the information stored in the user-motion storage portion andusing the input detection portion.
 10. The robot system according toclaim 9, wherein the user-motion storage portion is configured to storeinformation on a characteristic of each of a plurality of patterns ofmotion, wherein the motion detection portion is configured to identifythe motion of the input given by the operator, as one of the pluralityof patterns of motion, and wherein the control portion is configured toexecute an operation instruction associated with an identified patternof motion.
 11. The robot system according to claim 10, wherein theplurality of patterns comprises at least one of long touch, short touch,sequential touch, repeated touch, flick, swipe, pinch, and line drawing.12. The robot system according to claim 1, wherein the control portioncomprises an operation instruction storage portion configured to storean operation instruction that is executed when a corresponding motion isdetected by the motion detection portion, and wherein the operationinstruction comprises at least one of deceleration, acceleration, stop,restart, position control start, force control start, and error resetstart.
 13. The robot system according to claim 12, wherein theinstruction for the restart is an instruction to control the robot formoving a predetermined portion of the robot to a predetermined position.14. The robot system according to claim 13, wherein the predeterminedposition is a resume position, and the instruction for the restart is aninstruction to control the robot for resuming the robot from a state inwhich the robot is stopped due to an abnormal condition.
 15. The robotsystem according to claim 12, wherein the instruction for the forcecontrol start is an instruction to control the robot in accordance witha force value.
 16. The robot system according to claim 12, wherein theinstruction for the position control start is an instruction to controlthe robot in accordance with a position value.
 17. The robot systemaccording to claim 1, wherein the display portion is configured todisplay a motion-registration cancel button and a motion storage button.18. The robot system according to claim 1, wherein the display portionis configured to display a motion-registration-number display portionand a number forward/backward button.
 19. The robot system according toclaim 1, wherein the display portion is configured to display a motionedit button, and a motion-registration-number display portion blinks ifthe motion edit button is touched.
 20. The robot system according toclaim 19, wherein if a number forward/backward button is touched in astate where the motion-registration-number display portion is blinking,a motion corresponding to a motion registration number is displayed in asequential manner.
 21. The robot system according to claim 1, wherein,when a number forward/backward button is selected by the operator, thecontrol portion is configured to cause a display portion to display theinformation on the motion of the operator and the information on theoperation instruction associated with the motion of the operator in asequential manner.
 22. The robot system according to claim 1, whereinthe control portion is configured to acquire a feature values of a usermotion by using residual judgment after fitting by the least squaresmethod with a model waveform; or waveform pattern recognition usingmachine learning.
 23. The robot system according to claim 1, wherein, bythe operation of the operator, the control portion is configured tocause the display portion to display a list including the information onthe motion of the operator and the information on the operationinstruction associated with the motion of the operator.
 24. The robotsystem according to claim 1, wherein the control portion is configuredto cause the display portion to display the information on the motion ofthe operator, the information being provided with an arrow thatindicates how the information has been inputted by the operator.
 25. Therobot system according to claim 1, wherein the control portion isconfigured to cause the display portion to display a motion edit button,accept a selection of the information on the motion displayed on thedisplay portion, and accept edition of the information on the motion byaccepting an input of the motion edit button.
 26. A control devicecomprising: a motion detection portion configured to detect a motion byusing an input detection portion, the input detection portion beingconfigured to detect an input given from an operator to a robot body,the motion being given by the operator; and a control portion configuredto cause the robot body to execute an operation instruction associatedwith the motion detected by the motion detection portion, wherein, by anoperation of the operator, the control portion is configured to cause adisplay portion to display an information on the motion of the operatorand an information on the operation instruction associated with themotion of the operator.
 27. An information processing device comprising:a motion detection portion configured to detect a motion by using aninput detection portion, the input detection portion being configured todetect an input given from an operator to a robot body, the motion beinggiven by the operator; and a control portion configured to output anoperation instruction associated with the motion detected by the motiondetection portion, wherein, by an operation of the operator, the controlportion is configured to cause a display portion to display aninformation on the motion of the operator and an information on theoperation instruction associated with the motion of the operator.
 28. Acontrol method comprising: detecting an input given from an operator toa robot body; detecting a motion given by the operator; and causing therobot body to execute an operation instruction associated with thedetected motion, wherein, by an operation of the operator, causing adisplay portion to display an information on the motion of the operatorand an information on the operation instruction associated with themotion of the operator.
 29. A method of manufacturing a product by usingthe robot body that is controlled by the control method according toclaim
 28. 30. A computer-readable non-transitory recording mediumstoring a program that causes a computer to perform the control methodaccording to claim 28.